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(54) METHOD AND DEVICE FOR PICKING UP COLOR IMAGE 

(57)Abstract: 

PROBLEM TO BE SOLVED: To enable imaging rich in color reproducibility even in 
the case of photographing using single plate type and double plate type image pickup 
devices by converting an incident optical image to color separated images deviated just 
for fixed distances different for each color in the arranging direction of pixels of 
solid-state imaging device on the image pickup plane of this device. 
SOLUTION: A transparent flat plate 1 1 having a refraction factor (n) is placed 
obliquely to an optical axis, and this transparent flat plate 1 1 is installed while adjusting 

its incident angle θ and its thickness (t) so that the interval of refracted light beams ^ 

in three colors R, G and B can become a pixel pitch. For example, when the incident 
angle θ is 1 5°, the difference of every color deviation quantity, namely, the 
distance between R and G parts and the distance between the G and B parts are 
calculated to be 0.95* 10-3x(t). When the pixel pitch is 6.8μm, in the case of single 
plate system, the thickness (t) is set at 7. 1 6mm so that the distance of three primary 
colors in separated image can be made equal with the pixel pitch. Besides, since it is 
enough only to divide two colors R and B into adjacent pixels in the case of multi-plate 
system, it is enough only to set the thickness (t) to 3.58mm. Thus, color separation 
control is enabled corresponding to the pixel pitch of imaging device. 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 

[The tectimcal field to which invention belongs] this invention is [0002] which is a thing 
about the color picture picturizing method and equipment which were especially excellent 
in color-reproduction nature about the color image pck-up equipment which performs a 
photographic subject's color image pck-up using solid state image pickup devices, such as 

CCD series. . . fiU 

[From Prior art] The method which obtains a color picture usmg the color niter array 
which consist of a color filter element of a color is widely used for the video camera or the 
still camera Generally, the solid state image pickup device is used and the veneer formula, 
2 board type 3 board type, and the multi-board type are put in practical use from pomts, 
such as a size of equipment, quality of image, and cost, by such image pck-up equipment 
now By the veneer method, the photoelectrical transducer arranged by two-dimensional 
has the filter of the color of either of the three primary colors of an image pck-up system, 
and forms a pixel, and a photographic subject's sexual desire news is obtamed from the 
signal from two or more pixels from which it differs on an image pck-up side. That is, 1 
pixel of a color picture is constituted based on the signal from two or more pixels. Usmg 
color-separation prism and the image pck-up element of two sheets with 2 board methods, 
one sheet has become a luminance signal, for example, G pictures, one more sheet has 
become a chrominance signal, for example, R, and B pictures, and 1 pixel of a color 
picture is constituted from the image pck-up element for chrominance signals by two or 
more pixels like a veneer method. In order to ask for high resolution more, when 3 board 
methods or a multi-board type is also made only into one sheet as an object for 
chrominance signals and uses the image pck-up element of two or more [ remaining ] 
sheets as an object for luminance signals, 1-pixel drawing composition of the color pixel is 
carried out by two or more pixels like a veneer formula. 

[Problem(s) to be Solved by the Invention] As it indicates drawing 7 by the conventional 
method that it mentioned above, each pixel corresponds to the portion from which the 
photographic subject differed spatially, respectively. Therefore, interpolation processing 



was performed using the sexual desire news from the portion from which a photographic 
subject differs altogether in constituting 1 pixel of a color picture from two or more pixels. 
However, when interpolation processing was performed and a color picture was 
constituted, there was a fault of being easy to generate a color alias. Although interpolated 
by the chrominance signal called for not from interpolation of a simple chrominance signal 
but from a chrominance signal in USP 4,642,678 in order to reduce such a false color, 
since the chrominance signal used in case it asks for a chrominance signal was a 
chrominance signal from the position where it differs on a photographic subject, if the 
effect was enough in a place where brightness and a hue change a lot, there was nothing, in 
the case of the digital still camera which was suitable for the cellular phone especially, 
there are many veneer formulas, and since an output signal is used not as an animation but 
as a static image, such a false color poses a serious problem, and when color-printed, 
causing deterioration of the quality of image which does not bear for seeing also often 
comes out Even if this invention is made in view of such a problem and it picturizes using 
the image pck-up equipment of a veneer formula and 2 board type, it aims at acquiring the 
picture excellent in color-reproduction nature. 
[0004] 

[Means for Solving the Problem] In order to solve this fault in this invention, incidence is 
carried out to the solid state image pickup device which has the color filter array which 
consists of two or more colors the optical image formed of the optical image formation 
system. In the color picture picturizing method for carrying out signal processing of the 
signal acquired from each pixel, and obtaining a color picture signal, the optical image 
which carries out incidence on the image pck-up side of this solid state image pickup 
device It is made to change into the color-separation picture to which only a fixed distance 
which changes with colors in the direction of the list of the pixel of this solid state image 
pickup device was shifted, as signal processing From two chrominance signals from a 
photographic subject's same position, ask for a chrominance signal to the position of at 
least two places, and it is carried out based on this chrominance signal. It is supposed that 
it asks for the chrominance signal of a position which is different in two places with 
interpolation, this - further When the three primary colors of color picture image pck-up 
equipment were set to R, G, and B and the fixed amount of gaps by the color of the picture 
by which color separation were carried out [ aforementioned ] was expressed as a distance 
of the R section in a color-separation picture, the G section and the G section, and the B 
section, it was made for the aforementioned amount of gaps to become equal to the pixel 
pitch of this image pck-up element. Moreover, we decided to use equipment equipped with 
a means to realize the method. 
[0005] 

[The operation gestalt of invention] The separating-the color method and means of the 
invention in this application are shown in drawing 1 . before the color solid state image 
pickup device which has the color filter which consist of a color, to the optical axis of 
image formation optical system, it has an inclination, and a transparent plate is boiled and 
placed, the color is separated in the form which was, on the other hand, able to shift the 



beam of light from a photographic subject to the RGB color order at « and mcidence is 
caTed ouuo 4e adjoining piiel The color-separation picture pixel to which only a fixed 
dtocewhTch changes with colors in the direction of the lis. of me pixel of an image 
pck-up element was shifted by this is obtained. 

Sple 1 ] As shown in drawing! ; the transparent plate 1 1 of a refractive index n is 
1 , T h tn «n rmtiral axis and angle theta thickness t of the transparent plate 11 is 
tSS* rays uiterva. of R, G, and B3 color may become 

*&SSZ!h*K The amount h of gaps can be expressed by the following 

formulas. 

V^%*^1**^ here - * ° pticai rr of fte monotoI,ous 

g.ass qualify of the material SF 2 O^fi^f*^ m , As 1.661, if the 
«ot^^ 

degrees, i.e., the distance of the R section and the G section and die 

section and the B section are calculated, it will be set to hB-hG= hG-hR- 0.95x10 3xt. in 

mfcase p= 6 8 micrometers of pixel pitches, with a veneer method, it is 

t 6 8/0 95x10-3=7™ mm about monotonous thickness. Then, distance in three primary 

cirin *e inside of a decomposition picture can be made equal to a pixel pitch. In this 

casTthe dasTwhich has the distributed property of a desirable refractive mdex is used so 

She distlclof the R section, the G section and the G section, and the B section may 

"tS XtTs necessary is just to set thickness of the parallel monotonous glass 

E££3to3 58mm of the half by the multi-board method, since what * necessary is o 

«de orfy RB2 color into a contiguity pixel. Thereby, color-separation adjustment which 

suited the pixel pitch of an image pck-up element can be performed. 

[Sle 21 It is shown in drawing 2 . It places so that an interval may be prepared and 
Km of a refractive mo^nTy be opposed and the angle <"» d of 

S,s^^^^ 

RGB3 color is divided into an interval p, and it is [0008]. is good. A <^ cul ™ on 
is N example 1]. It ,s the same and me amount h of gaps ,s calculated by the 

following formulas. 

ttmZ'^^ooZ^m.) here. The color-separation image whose pixel 
ptehS^ce of the R section, me G section and the G section, and the B section 
suited is obtained by selecting theta and d using this formula. 

[Sple 31 The method of forming the color-separation picture by the acoustooptic 



deflector 13 was shown in drawing 3 . The amount of gaps a color-separation picture is 
acquired and according to the color is also a place [0010] by adjusting a driving signal V 
and an interval d. It is set as the value of a law. A calculation principle is [an example 1]. It 
is the same and the detail of calculation is omitted. Next, it sets below and is the method of 
this invention. The chrominance signal to depend: The interpolation chrominance-signal art 
using C=R/G is shown. 
[0011] 

[Example 4] The image pck-up element which has a stripe-like color filter in drawing 4 is 
shown. The number which met the periphery frame of drawing is a coordinate which 
shows the position of the pixel of an image pck-up element, using a pixel (i, j) as a display 
of a pixel position, i shall express a horizontal (the inside of drawing the direction of a 
horizontal axis) coordinate, and j shall express a vertical (the inside of drawing the 
direction of a vertical axis) coordinate. Image formation of the color-separation image 
which was able to shift the picture by the color horizontally (in direction which is a 
horizontal axis all over drawing) is carried out to the RGB part of an element to the image 
pck-up element which has this array, respectively. At this time, the chrominance signal of 
the point on a photographic subject is obtained as follows. For example, considering the 
point on the photographic subject with which image formation of the image of G color is 
carried out to the pixel which has G filter of eye two trains, the signal of B color is 
outputted from the pixel which has B filter of eye three trains by the pixel in which the 
signal of R color of this point has R filter of eye one train. Considering the point on the 
photographic subject with which similarly image formation of the image of G color is 
carried out to the pixel which has G filter of eye five trains, the signal of B color is 
outputted from the pixel which has B filter of eye six trains by the pixel in which the signal 
of R color of this point has R filter of eye four trains, namely, R signal of eye one train « R 
of G signal position of eye two trains — similarly B signal of eye three trains is set to B of 
G signal position of eye two trains Thus, the display of the chrominance signal from each 
pixel obtained is performed like G21 about R and B about G signal by making into a suffix 
the coordinate which is the pixel of G color which the point on the photographic subject of 
the point has by making into a suffix the coordinate of the pixel which has G filter. 
Therefore, although pixel positions differ, since G21 and R21 have the the same suffix, 
they express the chrominance signal from the same point on a photographic subject, (a) 
shows a stripe array color filter array among drawing 4 , and, as for the picture position of 
G component, (c), and (d), (b) shows the effective picture position of R and B component, 
respectively. 

[0012] One example of signal processing of this invention is carried out to interpolation of 
a chrominance signal using a chrominance signal using linear interpolation about a 
luminance signal. Therefore, interpolation using the chrominance of a linear 
interpolation:G31'=(2*G21+G51)/3G41'=(G21+2*G51)/3** chrominance signal of the ** 
luminance signal G : (for example, R) 

chrominance: -- C=R/GR31'=G31'* (2*(R21/G21)+R51/G51)/3 each chrominance C in 
R41'=G41"* (R21/G21) (+2*R51/G51) / 3 parentheses serves as a chrominance signal 



from a photographic subject same position It is calculated like [ B ] R. In addition, a prime 
SesseSlation data. When the test chart was photoed £ « : pc £ 
element usine such interpolation and havuig been outputted to the CRT screen ana me 
co™ tL beautiful picture in which a false color is not consp.cuous was acquired. 

Sple 5] The light filter shown in drawing shows the so-called Bayer array which 
»t2d ereater importance than to that of a chrominance signal to the resolution of a 
ScS The mLber which met the periphery frame of drawing ts a coordinate 
whTcnows A position of the pixel of an image pek-up element, using a pixel (, j) as a 
^ttv of a pkel position i shall express a horizontal (the inside of drawing the direction 
of a EffiaBW^rftato, and j shall express a vertical (the inside of drawing the 
fccti"n o?a vertcal axis) coordinate. Image formation of the color-separation nnage 
S was able to shift the picture by the color horizontally (in (hrection which . a 
ht^ZS over drawing) is carried out to the RGB part of an element to the image 
nck-up dement winch has this Ly, respectively. At this time, the chrommance signal of 
fhtZm ofa PhTographic subject is obtained as follows. For example, considering the 
potat o^ tne XoSc subject with which image formation of the image of G color is 
cS out t I fltpkd (0 0) which has G filter, the signal of R color of this point is 
acTule™frl Pixel (1 -0) which has R filter. Considering the pornt on the 

E to G filter. Therefore, although pixel positions differ, smce GOO and R00 have the 
fte same suffix they express the chrominance signal from the same point on a 
ohoto^pht subject. A^nong this drawing, (a) is the Bayer array itself and the picture 
poSf G component, (c), and (d) show [ (b) ] the effective picture position of R and B 

Z^eSTXpkfrfanother signal processing by the method of this invention .s shown 
°e w 1 1 Zpo^oA signa! as shown in « 
Wndlare in the position conditions of a pixel that it should interpolate. When, as tor one, 
*e pix" wS alls the pixel which should perform interpolation processing ; a ^vel 
ar,d a pemendicular direction does not have a measurement chrominance s gnal (for 
^ple when the pixel which adjoins perpendicularly [ the case of the mterpolanon 
Passing of R signal to the pixel (0 1) of" (a) and another ] or honzontally ha 



right. 

** It is R'l \=(Gl 1/4) X to the signal from chrominance interpolation drawing 5 (c). F (R, 
G) 

It considers as F(R, G) =(R00/G00) +(R22/G22) +(R02/G02)+ (R20/G20) here. Each 
chrominance signal in a parenthesis is calculated from the chrominance signal from the 
same position on a photographic subject. However, prime expresses interpolation data. It 
is calculated like [ B ] R. 
** each RGB3 color ~ linear interpolation 

It is referred to as G'21=(G1 1+G31+G20+G22)/4R'21=(R20+R22)/2B'21= (Bl 1+B31) / 2 

to the signal from drawing 5 (b), and is the same also about other pixels. 

** Put on each pixel position and perform color composition. When the test chart was 

photoed by the image pck-up element using such interpolation and having been outputted 

to the CRT screen and the color-print, the beautiful picture in which a false color is not 

conspicuous was acquired. 

[0015] 

[Example 6] Still more nearly another signal-processing example is shown. G signal carries 
out total chrominance interpolation about linear interpolation and RB to the signal from 
drawing 5 . 

G'21=(G11+G31+G20+G22) /4R'21=(G'21/2) x (R20/G20) (+ (R22/G22)) 

G'12=(G1 1+G13+G02+G22) /4R'12=(G'12/2) x (R02/G02) (+ (R22/G22)) 

Each chrominance signal in a parenthesis turns into a chrominance signal from a 

photographic subject's same position. However, prime expresses interpolation data. About 

B, it is calculated like R. When the test chart was photoed by the image pck-up element 

using such interpolation and having been outputted to the CRT screen and the color-print, 

the beautiful picture in which a false color is not conspicuous was acquired. 

[0016] In addition, as a means to perform such signal processing, there is DSP (digital 

signal processor), for example. 

[0017] 

[Effect of the Invention] As explained above, if whatever the formula it calculates 
Chrominance C using the sexual desire news from ** photographic subject's same spatial 
position, and interpolates it based on the value and compounds a color picture using the 
sexual desire news from ** photographic subject's same spatial position, the color picture 
with few false colors of it will become possible. 

[0018] furthermore, a G stripe RB check array, a primary color color difference array, the 
various color filter arrays of 6 square-shape lattice, etc. -- also receiving — a picture ~ 
since RB each color is spatially in agreement with G picture, respectively, maintaining the 
resolution of the luminance-signal G picture which governs resolution that is, inner at least 
2 colors of RGB3 color surely serve as information from a photographic subject's same 
spatial position, and false color generating of the color picture after an image processing is 
mitigated compared with the conventional method 

[0019] The above content cannot be overemphasized by that there is the same natural 
effect also to a complementary color type color filter array, in addition, drawing 6 -- the 



invention in this application -- a basis - it is the example of DSC using **** image pck-up 
equipment Although each portion is connected physically drawing, it is also possible for 
the image pck-up section and the processing section to be divided, and to transmit by 
archive media, such as a magnetic tape and a disk. For example, the signal from the image 
pck-up section is recorded on the direct archive medium, and it processes by transmitting 
to the processing section behind. The picture signal processed similarly can also once be 
recorded, in order to display later. Execution of signal processing uses a digital computer, 
and the programmed microprocessor and the hardware of exclusive use. 



[Translation done.] 
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OaS^^ixS. ^=S:i3%lWI«Rfi#tt2Wa«Gli 
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(4 

5 

*H«4>R. |5lt<3?iJ@OBfg^(±2?i!@«0G(t^{4 

^m*\i. Gft^lSfflLTJi. G7</^*#f*« 
itc^g^a^kLT. RfcBfcHLTtt.> 
^friWJiW^GfeWB^^SSr^^k L 

BSHMBilWri**. *i¥*WC**«>"^ 

7 4 7*Kflfe7 o^rH- ( b ) {±Gj£#Offlf££A 
t_ ( C ). (d) fi-tti-fitR. B«4W^a»B«ffl 1 

[0012] *#*o®§^wfc^*>£*^±- * 

©»Jgft^G<7)y^7?fS]: 
G31' = (2*G2i+G 5 i) /3 
G41 ' = (G21+ 2*G5i ) /3 

?a : C = R/G 

R31' =G3i' * (2* (R21/G21) +R51/G51) 
/3 

Ru' =G*i' * ( (R21/G21) +2*R 5 i/G5i) 
/3 

«m^J:OTXb^A~hSrSI^L. CRTlE&tf 
[00 13] 

SR^cofflSRwUSt*^"**"?* 1 '' B*fitl*>ffc5fc 
Utli. B* < i . J ) 114*^*1*1 (H+T 

tf5«tL-C#fe*l6. «itf. G7<iH'*fr*&B* 
(0, 0) CGfeOTflbS^SftSftt*^*^^**^ 

m (-U 0) .fcOftMl*. G7 4**** 

*i"6B*(2. 1) -fcOfcWit. 
G7^i^* : ff-f6ffl*^S***^ fcl ' T ' RtB 
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6 

nimzfct^tix. wi.tr. Goo^iacff^. « 

GookRaoJilllftglifl^^ 

H+w <a>tt<4*EW'W>t»0>T*9. <b)liG 
ftftoBftttZ* ( c ) . ( d ) Bjft# 

*5$B»BflMa«** , f. 

[0014] *m<r>iv£iz}:imm *&s*)£ttM 

6^£li*BW*&B3l^B*^ 2 

mht. vtott. *«a»M*iirt-**B*fc*¥. 
«tar. pi (a) cdb* (o.i) izmt Rmm 

i^sBStfw&fe********^ HH 

(obs (i.i) {zm&m^etimmnm&x'h 

B#<D&fi^9?n=^y*fl#£*fiiU 

15 (c) .fcijaHI^fcttU 

R' 11= (Gh/4) X F (R.G) 

ZZlZ, F (R.G) = (Rao/Goo) + (R22/G 22 ) + (R 

02/602) + (Rto/G 2 ») tSmi*IW«-7nS7- 

frO**. (it. T?4*XM±W8¥-9i*rt: B 
fcoi. rc t R k B»CU«S^* • 
® R G B 3 fe-etl -TffllSI . 

30 05 (b) iO^M^^t. 

G' 21= (Gn+G3i+G2o+G2i) /4 

R' 21= (R20+R22) /2 

B' 21= (B11 + B31) /2 

kU. flfiwB^-RIU'Ct>P«"C*S. 

RTBSaDf* 7 - 7' 'J y b CcaiTJ L/C k d ^ . fSfe^ 
[0015] 

40 mmme ] $ ^csiwi#«!a**isM*^'*"' 135 * 

G' 21= (G11+G31+G20+G22) /4 

R' 21= (G' 21/2) x ( (R 2 g/Gzo) + (R22/ 

G22) ) 

G' 12= (G11+G13 + G02+G22) /4 

R' lt = (G' n/2) x ( (R02/G02) + (R22/ 

G 22 ) ) 
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cRTBffiat^*5-ryyKcaj*LfcfcC5. (&£ 
«. «i«r, dsp (f<^^wm7t) 

[00 17] 

>AC5rH-»L. *«>tt&Sfc:LTliHL. -fLT 
©g^^coiil-S^tta* 1 6 tfOfetf 8 Sr ffl k vC * 7 -B 
ftfiO^^-BflOTTCifc 

[ 0 0 1 8 ] »C, GX h y 4 7R B TfflfiEJIL fllfefe 
MBA. 6»WB : fK*l«0*«e7 < /I^KJWcttL 

*>oo. RB^fetl-ttL-eiiffiiaiWfcGBftkHRLT 
\^h<nx\ r>i o rg b 3fe«ortftfc2feli#-rtt5* 

BfflWiaaw* 5 -B«ro«fe»£jWI*3*i.* . 

COO l9)ti±C0l*imm&?4 7 , <?)&7J>l"?TU 

-f izmx nmftMiiWjb&Mm? tx'i>%\\ 

H. H6Urt^Wcifcr<lt«U«*fflvtf:DSC 

hit. mmmk&m&mmztix . «*f— 7^7-' -r 



&LTfc£. afcJMSOI^L-aiBlfcfrS. Rene 

««$nfcB««*4twc«*f « feAtrv * ->/^ei* 

10 

[02 ] *wNzmh^mim*<r)^m'hz>7 
[03 ] *$rat*jit*fe4M!fifeatf*^Rrafctf 

[04] *JMBtffl^4Xh 7^7-^^7-7^/^ 
[ 0 5 ] *JWJ£jm >4<M •VKJf*)* 7 - 7 * /U? £ 

[06 1 *mzmi&iKt%&v>m%<7)mfo£Kt7a 
[07] ^mx'<r>m^±<mAt&mm : i i ±.x'^ 

11 • • • • j§<OT?t¥« 

12 • • ■ • ifHjT'JTTA 

13 ■ • • • ^wt/t^m? 
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